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Fourth Year Students 
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Fn cjmeertn 

Error Gradients in Neural Networks 

Problem 1 

Consider the neural network illustrated in Ffcj. 1 9 
ujith inputs Oc 1 - 3.1 and oc 2 - 2 .?. The output 
neuron has a b/narg Sigmoidal junction. The 
weights are - o.G , ur f - ~o. S , and ur -0.5. 

The desired output of the network is cL - o. 9 , 
and. the actual output is denoted bg s . 

(a) .Cal cu I ate the error e = d-S and the half 

sguare error <£ - -i- <? Ir > the network response. 

(b) Calculate the gradients of . defined in part La) > 
with respect to the weights is ^ and ur 2 ^ 

at the weight values given. 


resp e&tive! g ^ 


Facultg of 



Problem 2 


P-epeat Prob.l when the output 


n euron 


haS a. 



bipolar s i gmo i dal function. 


Problem 3 

Refer to tine neural network 
gradients of defined in 

to weights ur * T/r t ? and xu^ ? 
found to be 

i -0.13 '2± - - 

d w o * c) W f 


°S Tbe 

Prob. 1 , with respect 
respectively p are 


O. 52 


2L 


-o.Gs- 


at particular weight values. Determine tbe inputs 
and. x £ . 


Problem 4 

ln Prob . 3 , let ur f - 0.3" and w 2 
the value of u r o if e ? defined 
found to he - 0 . 5 2 - The output 
binary sigmoidal function . 


— — 0 . 5 . Calculate 
in Proh. 1 j is 
neuron has cu 


Problem S 

Repeat Prob . 4 if e is found to he Q.iS - 
Problem 6 

Can you solve Prob. 4 ujith e - o.S ? justify. 


Problem 7 

Consider the neural network illustrated m Fig. 2, 


u>itln inputs oc 1 = - 2.6 and x s - - 1*3 . The two 


output neurons have 

Jo in ary 

sigmoidal ^functions. 

The weights 

are : 




» ^ 

= -2.2 

, = -0.9 , 

ur r 0. ?. 
21 

’ Ki 

.= o. 35 

ur - ~o. kb 
v 02 

The desired 

outputs 

of the 

ne t work are d ^ « 0.51 


and d 2 - 1.21 ? and the corresponding actual 
outputs are denoted by S t and S 3 ? r es^ p ect i ✓ elj . 

(a) Calculate the errors e 1 = d 1 -S 1 and e 2 = d 2 - S 2 P 
and the half summed square error i = ± (ef-t e* ) . 

(b) Calculate the gradients o f C, defined in part (a) 3 
cvlth respect to the weights of the network, 
respectively , at the weight values given. 


-f 1 



problem 8 

Refer to the neural net work of F ig . 2 . At output 
neuron 1 , the error e ^ - d 1 -S 1 is 1.22 when the 



V 


desired output d 1 is 1.4-1. At output neuron 2 
the error e 2 - d 2 - s 2 is -o.81 when the desired 
output d 2 is -1.23 . The two output neurons have 
bipolar sigmoidal -functions . The weights are - 
w n - -o.d 2 , */ 2 = O. 87 , ur si - 0.76 , 

W S2 -- 0.65 , ur t - -O. S3 , 53 

La) Determine the inputs x i and x z . 


(h) Calculate the gradients of the half 


r>ed 


sgua 


error £ = ± (ej* + <?/ ) with respect to the weights 
of the network , respectively , at the weight values 
given . 


Problem 9 

Refer bo the neural network of Fig. 2. The output 
neurons have bipolar sigmoidal Junctions . The errors 
in the response signals s f and S 2 are — 0.8 and 
- 0.9 j respectively , for particular values of 
network inputs dc 1 and x 2 . The weights are : 

~ —1.21 ? vr i2 - 1-44 , ur i = 0.88 , 

w* = -0.75 , *r t z 2 - 0 .fi ? - -o. 4 

The gradients of the half summed sguare error £ 
with respect to w o1 and u/ o2 are found to he 
D L - 0,2 and = 0.288. Calculate the 

gradients of £ wuh respect to the remaining 
weights j res pectively . What are the values of inputs 
X 1 and oc 2 *? 


Problem IQ 

Consider the neural network illustrated i n Ra. 3. Tke 


sigmoi dal 

The response signals are s f - o.£ and s 2 - 0.1 , 
wi.th errors e 1 = o.3S and e 2 =-0.35? respectively 
The three inputs are constrained with a condition 
Xj = — 0C 2 - 2 x 3 > and the tuJo bias weights are 

cons t rained with a. condt ’h'on -t vr a = 1. He. 

weights 
output neurons are - 

v n s-O.S , W, a =.- O.s , »r ( = o.lt , 


of the connections between input and 


Junctions. 


output neurons have binary 


W 2 =: 0,3 , * r 3l ■= 0.2 , - - °' 6 

(a) determine the inputs x 1 . 9 x 2 ? and oc 3 9 and 
the bias weights . 7 tr and ■ 

-(b) Calculate the gradients of the halj summed 
sguare error with respect to all weights of 
the network at the weight values speeiji ed. 




Problem 'll 

Consider the neural network illustrated in /-]'£ • 4 - 
The neurons both the hidden and output lagers 

have binary sigmoidal functions . The inputs are. 

3Cy z: 2.2 and ac 3 — — 1.8. The hidden- to-output 

weights are : 

% = 0.6 S' , v/ 2i - -O.SS , ur i - O.tf 

and the input- to-hi dden weights are ' 

"n = 0-1 , *£, = 0.7 , = 0.6 , 

us - o.Q , vr . ~ o.S , ur - - <o. 6" 

27 ' °1 ' 02 

Calculate the gradients oj f the half sguare error 


all 


uieig. 


>hts 


at the weight va 


g the ne two rk ? 


oith respect to 
respectively , a 
the desired, response oj the network to be oj 
Value 0.3 . 


hues given. Assume 



f~‘fl ■ 4 : Neural network -for Prob ■ 11 



y Problem 1 2 

Consider the neural network illust rated in R T S. 

The neurons o£ Loth the hidden and output lagers 
have loinarg sigmoidal functions . The inputs are 
2^ : -3,2 j and ^ ~ 2. <? . The actual res ponse 
signals are denoted kg s 1 and S 2 , and the 
corresponding desired signals are 0.18 and 0.\ \ 2 
respectivelg . The hidden-to-output weights are: 


= .0.? 
ur -o.S 




w;, - °-S , Ki - °-b 

and. the input-ta-hidden weights are: 

- 0.85 , ur 2 - -0.3 , is 3l = o. 7 ?^ w 27 = -0.<i J 

ur — — o. G 3 w — (3-62. 

ol 02 

Calculate the gradients oj ? the halj summed square 
error with respect to all weights ? respectivelg ^ 



-+ t 


-fi 


Fig. S : Neural network for Proh. 12 



